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DETAILED ACTION 

EXAMINER'S AMENDMENT 

1 . An examiner's amendment to the record appears below. Should the changes 
and/or additions be unacceptable to applicant, an amendment may be filed as provided 
by 37 CFR 1 .31 2. To ensure consideration of such an amendment, it MUST be 
submitted no later than the payment of the issue fee. 

Authorization for this examiner's amendment was given in a telephone interview 
with Jonathan Berschadsky on January 13, 2011. 

Please amend claims 1,14, and 1 6 as follows: 

Claim 1 (currently amended): An image processing method of generating a display 
image of a virtual space including a virtual object consisting of at least one part, 
comprising: 

a first acquisition step of acquiring a position and an orientation of a viewpoint of 
an observer; 

a second acquisition step of acquiring a position and an orientation of a pointing 
device; 

a specification step of specifying a part that is included in a-the virtual object and 
is designated by the pointing device, based on positions of one or more parts included 
in the virtual object and the position and the orientation of the pointing device; 
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a calculation step of calculating a position of a list image to be laid out in the 
virtual space based on the positions of the viewpoint and the pointing device, wherein 
the calculated position of the list image is near the position of the pointing device and 
closer to the position of the viewpoint than that of the pointing device, wherein the list 
image represents information of the specified part; 

a layout step of laying out the list image at the calculated position in a-the virtual 

space; 

a virtual space image generation step of generating a-the display image of the 
virtual space, in which the laid out list image and the virtual object are included, based 
on the acquired position and orientation of the viewpoint; and 

a composition step of composing the generated display image of the virtual 
space and an image of a physical space seen in accordance with the position and the 
orientation of the viewpoint to display the composed image at a head mounted display 
mounted on the observer's head. 

Claim 14 (currently amended): An image processing apparatus for generating a display 
image of a virtual space including a virtual object consisting of at least one part, 
comprising: 

a first acquisition unit adapted to acquire a position and an orientation of a 
viewpoint of an observer; 

a second acquisition unit adapted to acquire a position and an orientation of a 
pointing device; 
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a specification unit of-for specifying a part that is included in a-the virtual object 
and is designated by the pointing device, based on positions of one or more parts 
included in the virtual object and the position and the orientation of the pointing device; 

a calculation unit for calculating a position of a list image to be laid out in a-the 
virtual space based on the positions of the viewpoint and the pointing device, wherein 
the calculated position of the list image is near the position of the pointing device and 
closer to the position of the viewpoint than that of the pointing device, wherein the list 
image represents a list of pieces of information about one or more parts, included injhe 
virtual object , noar tho pos i t i on of tho po i nting dov i co ; 

a layout unit adapted to lay out the list image at the calculated position in the 
virtual space; 

a virtual space image generation unit adapted to generate a-the display image of 
the virtual space, in which the laid out list image and the virtual object are included, 
based on the position and the orientation of the viewpoint; and 

a composition unit for composing the generated display image of the virtual 
space and an image of a physical space seen in accordance with the position and the 
orientation of the viewpoint to display the composed image at a head mounted display 
mounted on the observer's head. 

Claim 16 (currently amended): A non - trans i ont non-transitory computer-readable 
storage medium having a program stored therein, that when executed by a computer 
causes the computer to perform an image processing method of generating a display 
image of a virtual space including a virtual object consisting of at least one part, the 
method comprising: 
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a first acquisition step of acquiring a position and an orientation of a viewpoint of 
an observer; 

a second acquisition step of acquiring a position and an orientation of a pointing 
device; 

a specification step of specifying a part that is included in a-the virtual object and 
is designated by the pointing device, based on positions of one or more parts included 
in the virtual object and the position and the orientation of the pointing device; 

a calculation step of calculating a position of a list image to be laid out in the 
virtual space based on the positions of the viewpoint and the pointing device, wherein 
the calculated position of the list image is near the position of the pointing device and 
closer to the position of the viewpoint than that of the pointing device, wherein the list 
image represents information of the specified part; 

a layout step of laying out the list image at the calculated position in a-the virtual 

space; 

a virtual space image generation step of generating a-the display image of the 
virtual space, in which the laid out list image and the virtual object are included, based 
on the acquired position and orientation of the viewpoint; and 

a composition step of composing the generated display image of the virtual 
space and an image of a physical space seen in accordance with the position and the 
orientation of the viewpoint to display the composed image at a head mounted display 
mounted on the observer's head. 
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Conclusion 

2. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to DAVID T. WELCH whose telephone number is 
(571)270-5364. The examiner can normally be reached on Monday-Thursday, 8:00- 
5:30 EST, and alternate Fridays, 8:00-4:30 EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Xiao Wu can be reached on (571)272-7761 . The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/dtw/ 

/XIAO M. WU/ 

Supervisory Patent Examiner, Art Unit 2628 



